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ABSTRACT 



This report summarizes research supports addressing three 
areas as targets for educational improvement in Alabama: beginning reading, 
expansion of reading power, and intensive intervention for those who are 
reading below grade level. The paper also describes Alabama's statewide 
initiative to implement best practices in these areas, and summarizes the 
results of the reading improvement imitative for the year academic 1999-2000. 
The Alabama Reading Initiative (ARI) , a collaborative model, was based on an 
extensive literature review and teacher training in summer institutes to help 
teachers implement research-based reading instruction. Implementation began 
at 16 schools named as Literacy Demonstration Sites for the 1998-1999 school 
year. In the second year of implementation, the year of this study, 
approximately 27,700 students, 2,354 teachers, 81 principals, 75 reading 
specialists, 64 higher education partners, and 21 preservice teacher 
education programs participated. The year- 2 evaluation involved internal and 
external evaluation components that included surveys completed by 1,860 
teachers, 11 principals, 74 reading specialists, 44 higher education 
partners, and 49 higher education faculty members. Overall findings include 
small but positive improvement on Stanford 9 reading test results for ARI 
students. Teachers at ARI schools reported increases in the use of 
research-based strategies and in student reading time, greater confidence and 
enthusiasm toward instruction, and improved student and teacher attitudes. 
Schools varied in the improvement noted on the Stanford 9 results, and 
findings suggested three factors linked to a greater degree of success: (1) 

the principal as a champion of ARI; (2) a hands-on and helpful reading 
specialist; and (3) a deeply involved higher education partner. (Contains 18 
references.) (SLD) 
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According to the report of the National Assessment of Educational Progress - NAEP 
(Williams, et. al. 1995), 40% of fourth graders, 30% of eighth graders and 25% of twelfth graders 
were reading below grade level. NAEP results also show that in Alabama 44% of fourth graders 
and 34% of eiglitli graders are scoring below the basic level in reading. Scores on the Stanford 
Achievement Test 9 given to Alabama public school students in Grades 3 through 12 in April 
2000 reveal that 127,626 students (30.39%) scored in the bottom four stanines. 

Knowing that reading is fundamental in preparing students for success in school as well 
as in life, tlie Alabama State Board of Education in 1996 appointed a group of educators from 
K-12 and liigher education, individuals from business and industry, and representatives from 
grassroots support groups to study the reading problem. This group known as the Alabama 
Reading Panel studied the research on reading and identified the following three areas that needed 
to be targeted to improve reading at the K-12 levels of instruction; beginning reading, expansion 
of reading power, and intensive intervention for those who are reading below grade level. This 
article will summarize the research that supports addressing these three areas, describe the 
statewide initiative to implement best practices in these areas, and summarize results of the 
initiative fortlie year 1999-2000. 

THE RESEARCH 

Beginning Reading 

Effective beginning reading instruction is balanced. In other words, it is comprised of 
explicit and systematic skill instruction in phonemic awareness and phonics in combination with 
many opportunities for authentic reading and writing. (Adams, 1995; Adams & Bruck, 1995; 
Boyer, 1996; Clay, 2001; Cooper & Kriger, 2001; Ruddell, 1999). This balanced approach, 
supported by the research is reflected in more detail by Snow, Bums, & Griffin (1998) who state. 
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Adequate initial reading instruction requires a focus on using 
reading to obtain meaning from print; understanding the 
sublexical structure of spoken words; exposing the nature of tlie 
orthographic system; practice in the specifics of frequent, regular, 
spelling-sound relationships; and frequent and intensive opportunities 
to read. (p. 223). 

According to Weaver (1994), “studies investigating children’s definitions of reading have 
found, too, that poorer and younger readers tend to conceptualize reading as a matter of decoding 
and getting words, whereas older and more proficient readers generally conceptualize reading as 
more a matter of understanding tlie text” (p. 3). Tlierefore, the issue of balance between skills- 
based instmction and authentic reading is important when teacliing children how to read. 
Expansion of Reading Power 

Expansion of reading power results when students learn strategies to help them to better 
comprehend text that tliey are reading. According to the Standards for the English Language Arts 
(1996), “In addition to the knowledge of text and text features, students need to learn an array of 
processes and strategies for comprehending and producing texts” (p. 16). In addition to strategy- 
based instruction, other factors that help to expand one’s reading ability include an increase in the 
amount of reading, vocabulary development, deep discussion and questioning, and connecting 
reading and writing 

Research shows that the 90'*’ percentile fiftli grader reads about 200 times more text per 
year than the 10'*' percentile reader (California Department of Education, 1996) and that there is a 
positive relationship between tlie amount of voluntary reading and gains on standardized reading 
achievement tests (Pearson, 1993). In addition, teachers are spending inadequate amounts of time 
for strategy-based, comprehension mstruction and still assigning reading and asking questions 
that require very short answers (Vacca & Vacca, 2002). Research also indicates that vocabular}' 
development is directly related to how much students read and to direct teaching in meaningful 
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contexts (Scott and Nagy, 1997; Beck & McKeown, 1991; Stanovich, 1993-94). According to 
Pearson (1993) “students who interact daily with print, read what others have written, and write 
to others regularly come to value reading as tools for learning, enjoyment, and personal insight” 

(p. 509). 

Effective Intervention 

Effective intervention can occur when students who read below grade level are provided 
with specialized instruction. Early diagnosis and early intervention is best (Walker, 2000; Cooper 
& Kriger, 2001). This cannot be accomplished witliout teachers who know how to become 
diagnostic teachers and who can accelerate instruction (Walker, 2000). During reading 
instruction tlie teacher must determine whether print or meaning or both are interfering with 
readmg and then must make the instructional adjustments that will promote student success, and 
must evaluate reading growth as a result of that instruction. Walker (2000) points out that the 
diagnostic teacher is both a teacher and an evaluator. She is aware of changes in reading behavior 
and tlie instructional adjustments that produced the change. “She focuses on teaching rather than 
testing to reveal those instructional adjustments that produce reading change for a particular 
student” (p. 122). Snow et al.(1998) state, “It is imperative tliat teachers at all grade levels 
understand... the role of instruction in optimizing literacy development” (p. 9). 

THE ALABAMA READING INITIATIVE 
Alabama Reading Initiative - The Plan 

In order to make a difference m reading achievement in Alabama, the Alabama Reading 
panel knew that “the nature and quality of classroom literacy instruction are a pivotal force in 
preventing reading difficulties in young children” (Snow et al., 1994, p. 223). Since 
knowledgeable teachers could be a the key to change, it became unportant to establish guidelines 
so that a minimum of 85% of teachers and their administrator in each school would receive at a 
Summer histitute two weeks of mtensive traiihng in readmg research and its practical application 
in tire three targeted areas of 1) beginning reading for kindergarten and first-grade students, 2) 









expansion of reading power for students in Grades 2 through 12, and 3) intervention for 
struggling readers at all grade levels. The plan, known as the Alabama Reading Initiative (ARI), 
is a statewide effort to improve reading instmction significantly and for K-12 public school 
students ultimately to achieve 100% literacy. 

The Alabama Reading Initiative Process 

The ARI seeks to accomplish its goals through the implementation of research-based 
reading instruction described above at Literacy Demonstration Sites. Tliese sites were selected by 
State Department of Education staff members and designees who traveled the state, visiting 
schools that had applied to participate in the program. All public schools in the state are eligible 
to become Literacy Demonstration Sites. Selection is based on the candidate school’s knowledge 
of ARI’s goals and program content and the school’s commitment to professional development in 
pursuit of 100% literacy. The Summer Institutes are presented regionally by ARI-trained readmg 
professionals using learning modules developed by ARI. 

Unique elements of the ARI model include 1) the tliree targeted areas cited above, 2) the 
all-inclusive K-12 focus, 3) the requirement that a minunum of 85% of the faculty attend the 
training, 4) the requirement that tlie principal participate in the same training as the faculty and 
lead the faculty during 10 hours of faculty meetings during the Summer Institute, and 5) tlie 
formation of partnerships between the schools and professional educators from Alabama’s 
Institutions of Higher Education . 

The Alabama Reading Initiative and the Higher Education Partnership 

The ARI is a collaborative model in which Higlier Education faculty have been partners 
from the outset. Tliese Higher Education professionals have served on the Alabama Reading 
Panel, helped to develop instructional modules for die Summer Institutes, trained both the 
presenters and their trainers for the Suimner Institutes, and presented learning modules at the 
histitutes. Tliey also participated in the Advanced Study Group, which during Year 1 was 
dedicated to reviewing current research and practice in-depth. An important by-product of the 
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active involvement of partner Higher Education faculty in ARI is the opportunity for alignment of 
pre-service reading education witli the research-based, professional development provided at the 
Summer Institutes. 

Implementation of the Alabama Reading Initiative 

Implementation of ARI first took place at 1 6 schools named as Literacy Demonstration 
Sites for the 1998-1999 school year. For the purpose of this study, these sites are grouped as 
Cohort A. Sixty-five additional schools beccune Literacy Demonstration Sites for the 1999-2000 
school year, grouped as Cohort B. 

Grassroots support for the ARI model has attracted substantial private funding for the 
Summer Institutes as well as for otlier key elements of the program. During ARI’s first year, 
1998-1999, no financial resources were available to support involvement of higher education 
partners; tlierefore, their participation was entirely on a voluntary basis. During the second year of 
ARI, 1999-2000, legislative funding was provided for partner Higher Education faculty to visit 
the 81 Sites on a monthly basis. State funding also was provided for 75 full-time reading 
specialists (trained through ARI) to serve tlie 81 Sites. Each reading specialist supported ARI 
implementation by working on a daily basis at tlie Literacy Demonstration Site to help struggling 
readers and to coach faculty relative to implementation of Summer Institute learning. 

In its second year of implementation, approximately 27,700 students, 2,354 teachers, 81 
principals, 75 reading specialists, 64 liigher education partners, and 21 pre-service teacher 
education programs in Alabcuna’s higher education institutions were directly involved in the ARI. 
Of the 8 1 Literacy Demonstration Sites participating in the second year of the ARI (Cohort A and 
Cohort B together), five were primary schools; 56 were elementary schools; nine were middle 
schools; four were liigh schools; three were K-12 schools; tluee were K-8 schools; and one was a 
K-7 school. Twenty-four schools were from small school systems (3,900 students or less), and 57 
schools were from systems classified by the Alabama State Department of Education as large 
(more than 3,900 students). School populations ranged from 188 students to 1,153 students. 
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Socioeconomic status, as measured by the percentage of students receiving free lunch, ranged 
from 2.56 to 99.10 percent. In 30 of the 81 schools, more tlian half of the students received free 
lunch. 

THE EVALUATION PROJECT 

Tlie ARI evaluation was frinded by a federal grant under the U.S. Department of 
Education Dwight D. Eisenliower Professional Development Program (P.L. 103-382) 
administered by the Alabama Conmiission on Higher Education. The evaluation was furtlier 
guided by mput from an Evaluation Oversight Committee composed of representatives from 
private industry, Alabama Coirunission on Higher Education, the Alabama State Department of 
Education , institutions of liigher education, tlie At- Foundation, and Literacy Demonstration Sites 
(central office administrators amd building principals). 

This evaluation was approached from an intemal/extemal perspective. The external 
evaluation Project Director from the University of Alabama at Birmingham Center for 
Educational Accountability, was responsible for data analysis and interpretation to ensure the 
integrity of the findings. Tlie internal evaluation Project Director from the University of Alabama 
at Huntsville, and representatives from the ARI staff participated in the design, development, and 
execution of the evaluation. 

Purpose 

The evaluation of Year 2 was crafted to examine the Alabama Reading Initiative from 
two perspectives. In the K-12 Component, ARI is evaluated for the purpose of improving its 
effectiveness. In the Pre-Service Component, ARI is evaluated for the purpose of determining its 
impact on pre-service instruction. The Year 1 evaluation was of the first sixteen Literacy 
Demonstration Sites’. During Year 2, 81 schools were involved. 



See “Evaluation of the Alabama Reading Initiative 1998-1999,” prepared for the Alabama Conunission on Higher 
Education by the Center for Educational Accomitability, University of Alabama m Birmingham, Birmingham, AL. 
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The K-12 Component 

A primary purpose of the Alabama Reading Initiative (ARI) Year 2 evaluation was to 
gain an understanding of tlie factors that could be linked to increased student achievement in 
reading. With this understanding, ARI could adjust its requirements and procedures to reach its 
goals of 100% literacy and of expanding to all schools in the state. In order to achieve successful 
statewide expansion, the ARI needed to know which activities, requirements, and/or features are 
essential and which are optional; which are strong and which are weak; wliich should be 
replicated and which should be adjusted. 

Evaluation of Tlie K-12 Component was therefore embodied in four vital questions: 

(1) To what extent are ARI schools making progress toward 100% literacy? 

(2) Wliich ARI schools are makmg progress toward 100% literacy and which are not? 

(3) What factors are related to school outcomes? 

(4) Wliy are some ARI schools maknig more progress than others? 

Tlte Pre-Service Component 

The long-range solution to Alabama’s'pursuit of 100% literacy lies, at least in part, with 
teacher preparation programs. Teacher preparedness to instruct all public school students to learn 
to read, to expand tlieir reading power, and to provide intervention for struggling readers depends 
in part on tlie preparation of teachers and administrators hi pre-service programs. 

Two questions were posed in evaluating the pre-service component: 

(1) To what extent are tlie elements of ARI reflected in pre-service teacher education 

programs tliroughout Alabama? 

(2) Wliat ARI factors are related to change in pre-service teacher education programs? 
Instruments and Data Collection 

Both achievement data and survey responses were used for the K-12 component of the 
evaluation, and survey responses were used for tlie pre-service component. Achievement data 
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used in tlie evaluation were provided by the Alabama State Department of Education and 
included results from the Stanford Achievement Test, Ninth Edition for students in Grades 3-11. 

Survey instruments were developed collaboratively by project directors, with feedback 
from the Oversight Committee. Both rating-type and open-ended items were included in 
instniments distributed to teachers, principals, reading specialists, higher education partners, and 
higher education reading faculty. Responses from teachers, principals, reading specialists, and 
higher education partners comprised survey data for the K-12 component. Responses from higher 
education partners and higher education reading faculty comprised data for the pre-service 
component. 

Data collection relative to the survey instruments was accomplished through regional 
meetings with school personnel and througli mailings to higher education partners and to deans of 
schools of education. Returned surveys included representation from all 81 schools for at least 
one of the surveys. Responses (response rates) included 1,860 teachers (86%), 77 principals 
(95%), 74 reading specialists (91%), and 44 higher education partners (54%). Surveys also were 
returned by 49 higher education reading faculty. 

Results 
Question 1: 

To what extent are ARI schools making progress toward 100% literacy? 

Answer to Question 1: 

ARI schools made more progress toward 100% literacy than did non-ARI schools. 

• Improvements in Nonnal Curve Equivalency" (NCE) scores across Reading Comprehension, 
Reading Vocabular>', and Total Reading between 1998 and 2000 averaged 1.05 for Cohort A 
ARI schools and .24 for non-ARI schools (all other Alabama public schools). Improvements 
in NCE scores across all these reading subtests between 1999 and 2000 averaged .28 for 

‘ An NCE score is similar to a percentile score in that it can take values between 1 and 99. Unlike percentile scores, 
NCE scores can be averaged, allowing means to be calculated across schools and across grade levels. 
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Cohort B ARI schools and .20 for non-ARI schools. These improvements translated into 
small but positive differences in effect sizes^ favoring ARI schools for each reading subtest 
for Cohort A and B. 



NCE Means and Effect Sizes: Cohort A and 



^on-ARI Schools 



Cohort A Schools 



Non-ARI Schools 



Subtest 


1998 

Mean 

NCE 


2000 

Mean 

NCE 


Differ 

ence 


Effect 

Size 


1998 

Mean 

NCE 


2000 

Mean 

NCE 


Differ 

ence 


Effect 

Size 


Total Reading 


51.71 


52.67 


0.96 


0.05 


49.69 


49.87 


0.18 


0.01 


Reading Vocabulary 


51.37 


53.10 


1.73 


0.09 


49.94 


50.48 


0.54 


0.03 


Reading Comprehension 


52.02 


52.47 


0.45 


0.02 


49.74 


49.73 


-0.01 


-0.00 


NCE Means and Effect Sizes; Cohort B and ' 


Non-ARI Schools 




Cohort B Schools 


Non-ARI Schools 


Subtest 


1999 


2000 


Differ 


Effect 


1999 


2000 


Differ 


Effect 




Mean 


Mean 


ence 


Size 


Mean 


Mean 


ence 


Size 




NCE 


NCE 






NCE 


NCE 






Total Reading 


52.70 


52.97 


0.27 


0.01 


49.68 


49.87 


0.19 


0.01 


Reading Vocabulary 


52.68 


53.04 


0.36 


0.02 


50.22 


50.48 


0.26 


0.01 


Reading Comprehension 


52.48 


52.68 


0.20 


0.01 


49.59 


49.73 


0.14 


0.01 



• As a group, ARI schools decreased the population of “struggling readers” (students scoring 
in the bottom three stanines on the Stanford 9 reading subtests) by as much as 10%. Parallel 
reductions in the percentage of struggling readers in non-ARI schools over the same periods 
of time were less than half that of the ARI schools; 





1998-2000 Decrease 


1999-2000 Decrease 


Subtest 


Cohort A 


Non-ARI 


Cohort B 


Non-ARI 


Total Reading 


9.46% 


1.81% 


7.83% 


2.18% 


Reading Vocabulary 


10.19% 


4.68% 


6.39% 


3.01% 


Reading Comprehension 


8.45% 


(.09%)* 


6.56% 


1.42% 



*Increase 



3 

Effect sizes are indices of the practical significance of differences between average scores. They are determined by 
computing the differences between means for two groups and then dividing the difference by the amount of dispersion 
in the scores (standard deviation). Effect sizes may be positive or negative. 
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• ARI schools increased the percentage of “grade-level re ers” (students scoring in the top 
five stanines) more than non- ARI schools over the samt eriods of time. ARI Cohort A 



schools increased at rates that ranged from 2. 1 8% to 4. % as compared to rates of .72% to 

2.3 1% for non-ARI schools. The rate of increase for C ort B schools ranged from 1 .28% to 
1 . 84% as compared to rates of increase for non-ARI sc ols that ranged from . 87% to 1.01%. 



Increases in the Population of Gr e-Ievel Readers 





1998-2000 Increase 


1999-2000 Increase 


Subtest 


Cohort A 


Non-AR] 


Cohort B 


Non-ARI 


Total Reading 


2.18 


1.. i 


1.53 


.90 


Reading Vocabulary 


4.56 


2 .: ' 


1.28 


1.01 


Reading Comprehension 


2.66 




1.84 


.87 



Question 2; 

Which ARI schoob are making progress toward 100% literacy and which are not? 

Answer to Question 2; 

Findings support that the vast majority of ARI schoob are making progress toward the goal 
of 100% literacy. Still, there is considerable variability in the gains made by ARI schoob. 

Approximately 70 percent of ARI schools demonstrated small gains (effect sizes greater 
than 0 and less than .40) on Reading Vocabulary, Reading Comprehension, or Total Reading; and 
an additional five percent of ARI schools made moderate (effect sizes between .40 and .70) or 
large gains (effect sizes of greater than .70) on these measures. An example of the trends in 
performance across schools is seen in Figure 1. This figure graphically displays the range of 
change in the percentage of “grade-level readers” (i.e., students scoring at or above stanine 5 on 
the Reading Comprehension subtest) for Cohort A and Cohort B. The figure reveals that while 
the majority of schools in Cohorts A and B made positive changes, some schools do particularly 
well and some schools do less well in terms of the indicator. Similar ranges of performance 
between schools were found for all indicators of literacy. 
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Because of the variability illustrated in Figure 1, it was important to identify factors that 
might account for differences in the performance of ARI schools. Efforts to identify such factors 
are summarized later in Questions 3 and 4. 



Figure 1. Change in Percentage of Students in Cohort A and Cohort B Schools 
Scoring at or above Stanine 5 on the 
Stanfords Reading Comprehension Subtest 




— represents scores from 15 Cohort A schools 
represents scores from 61 Cohort B schools 



Other indicators of positive, yet variable progress among ARI schools include tlie 

following; 

• Stanford 9 scores revealed positive progress toward 100% literacy on one or more of the 
reading subtests (effect sizes ranging from .01 to .72) in 13 of the 15 Cohort A schools that 
have Stanford 9 scores and 45 of the 61 Cohort B schools. 

• Decreases in the percentage of struggling readers (.02% to 18%) occurred in 10 of the 15 
Cohort A schools and 47 of the 61 Cohort B schools on one or more of tlie reading subtests. 

• Increases in the percentage of students scoring “on grade level” (.05% to 28%) occurred 
m 13 of the 15 Cohort A schools and 5 1 of the 6 1 Cohort B schools on one or more of the 



reading subtests. 



• Systematic and substantial decreases in discipline and special education referrals were 
found in tlie subset of ARI schools (7 in Cohort A and 27 in Cohort B) that reported such 
data. Discipline referrals in the reporting Cohort A schools decreased by 67%, from 1,795 
referrals in 1998 to 596 referrals in 2000. Tlie average decrease in discipline referrals across 
the 27 reporting Cohort B schools was 23%. 

• Asa group, the eight reporting Cohort A schools decreased special education referrals by 
28% from 1998 to 2000. Tlie 33 reporting Cohort B schools decreased special education 
referrals by 14% from 1999 to 2000. 

To understand why some schools demonstrated marked improvement and otliers showed 
losses in acliievement, subsets of higher- and lower- performing schools were identified. The 
method used for identifying higlier- and lower- achieving ARJ schools included approximately 26 
improvement indicators from the Stanford 9. Schools included in the final subsets of higher- and 
lower-perfonning ARI schools were tliose that demonstrated substantial consistency across these 
improvement indicators. Seven Cohort A schools were identified as higher-performing schools 
and two Cohort A schools were identified as lower-performing. In Cohort B, 14 schools were 
identified as higher-performing and eight were identified as lower-performing. 

Following identification of higher and lower performing schools, Stanford 9 results for 
all schools in each group were analyzed. Analyses included the percentage of students scoring in 
stcinines 1 tlirough 3, the percentage of students scoring at or above stanine 5, and NCE means 
and effect sizes. Results for Total Reading, Reading Vocabulary, and Reading Comprehension 
are summarized in the tables below. 





13 



Percentage Change: Students Scoring in Stanines 1-3 



Subtest 


Percentage Change 




Cohort A 
High 


Cohort A 
Low 


Cohort B 
High 


Cohort B 
Low 


Total Reading 


-13.35 


20.40 


-24.83 


29.19 


Reading Vocabulary 


-7.01 


’ 30.39 


-24.06 


27.90 


Reading Comprehension 


-12.26 


9.43 


-20.38 


29,90 



Percentage Change: Students Scoring at or above Stanine 5 



Subtest 


Percentage Change 


Cohort A 
High 


Cohort A 
Low 


Cohort B 
High 


Cohort B 
Low 


Total Reading 


2.15 


-3.01 


11.56 


-6.54 


Reading Vocabulary 


5.46 


-9.51 


10.85 


-5,51 


Reading Comprehension 


3.29 


4.16 


10.29 


-6.74 



Mean NCE Difference Effect Sizes 



Subtest 


Effect Sizes 


Cohort A 
High 


Cohort A 
Low 


Cohort B 
High 


Cohort B 
Low 


Total Reading 


0.06 


-0.02 


0.16 


-0.16 


Reading Vocabulary 


0.10 


-0.13 


0.18 


-0.15 


Reading Comprehension 


0.04 


0.04 


0.13 


-0.15 



Clearly, differences exist between higher and lower performing groups. Although 
Reading Comprehension, Reading Vocabulary, and Total Reading are highlighted here, they were 
not the only subtests used to identify higher and lower performing schools, as noted earlier. 

Other subtests included Total Matli, Total Language, Science, Social Science, and Total Battery. 
Wliereas liigher performing schools performed better than lower performing schools across all 
three reading subtests, tlie most obvious difference between higher and lower performing ARI 
schools was the schools’ ability to reduce the percentage of struggling readers in the school. Tliis 
reflects ARI efforts to improve reading skills among struggling readers (students scoring in 
stanines 1-3). 
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Questions 3 and 4: 



What factors are related to school outcomes? 

Why are some ARI schools making more progress than others? 

Answer to Questions 3 and 4: Several factors discriminated higher- and lower-performing 

ARI schools. The leadership of the principal and the helpfulness of the reading specialist 

had the greatest impact on student achievement. 

• The Principal as ARI Champion . Other tlian the ARI training itself, data point to the 
principal as having tlie greatest impact on student achievement. Correlations'^ between 
principal leadership scores as provided by teachers, higher education partners, and reading 
specialists and Stanford 9 effect were .20 or greater between leadership and one or more of 
the reading scores. In higher-performing ARI schools, the principal supported 
implementation by facilitating ongoing professional development; providing resources emd 
materials; adjusting schedules in order to enhance reading instruction; supporting and 
monitoring teacher implementation in the classroom; and encouraging faculty members. In 
lower- performing schools little or no support was noted. 

• The Hands-On. Helpful Reading Specialist . The reading specialist also made a difference. 
Tlie large majority of teachers in botli liigher-performing and lower-performing schools 
considered the role of the reading specialist as important to the successful implementation of 
ARI. Seventy percent (70%) of the teachers expressed in surveys that the reading specialist 
was important to serving the needs of students and teachers. However, teacher and principal 
ratings of tlie helpfulness of the reading specialist were greater in liigher-performing schools 
than in lower-performing schools. Correlations in excess of .20 were found between 
teachers’ ratings of tlie helpfulness of tlie reading specialist and the scores for Reading 



Correlations are statistical values that range from -1.0 to 1.0. They relate one score to another score. For 
example, if scores on principal leadersliip are positively correlated with scores on the Stanford P, we can 
conclude that increases in principal leadership are associated with gains on the Stanford 9. The liigher the 
correlation is, the greater is the association between leadersliip and gains. 
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Comprehension and Reading Total on the Stanford 9. Teachers reported tliat tlie reading 
specialist impacted reading skills of struggling readers by working with them directly and by 
encouraging and assisting teachers. Teachers also conunented on tlie help that reading 
specialists provided through program oversight, professional development, and materials and 
resources, 

• The Deeply Involved Higher Education Partner . Differences existed in higher- and lower- 
perfonning schools in tlie perceived lielpfLilness of the higher education partner, particularly 
as evaluated by the reading specialist. In higher-perfonning schools, reading specialists 
report that higher education partners had direct involvement with students, teachers, and 
reading specialists. Tliey conducted professional development sessions; modeled instruction; 
arranged for workshops; worked with struggling readers; worked one-on-one with teachers in 
their classrooms; trained new teachers in the ARI modules; and consulted with tlie principal. 
Higher education partners in lower- performing schools were reported to serve more as an 
emotional support. They hstened and encouraged but were less likely to be reported as 
offering subject matter-related expertise, demonstrations, consultation, and problem-solving. 

• Other Implementation Influences . Other factors discriminating higher- and lower- 
performing schools were teacher reports of the rates at wliich ARI components were 
implemented, ongoing professional development hours, increased attention to student reading 
time, greater attention to reading instructional strategies, more ongoing assessment of student 
progress, and increased focus on struggling readers. 

In addition, teachers responding to survey data reported that ARI had positive impacts on 
tlieir teacliing and student learning. Changes in teacliing included use of research-based 
strategies, increases in time dedicated to student reading, increased confidence and enthusiasm 
toward reading instruction, increased awareness of struggling readers, and positive school-wide 
change in the learning environment. Changes reported in student learning included unproved 
reading skill, improved writing abilities, and increased motivation to read. 
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Questions 5 and 6: 

To what extent are the elements of ARI reflected in pre-sen i teacher education programs 
throughout Alabama? 

What ARI factors are related to change in pre-service teach education programs? 

Answer to Questions 5 and 6: 

Findings indicate that changes in course content occurred in re-service teacher education 
programs throughout Alabama as a result of the ARI. Sur% data suggest further that 
teacher education faculty members perceived the ARI to bei It their teaching and their 
pre-service students. 

• All higher education partners and reading faculty who respi led to surveys indicated 
changes in course content that reflected material contained i he ARI teacher training 
modules and other ARI-published documents. 

• All liigher education partners and reading faculty who resporded to surveys indicated tliat 
current course syllabi incorporated the new reading standards adopted by the Alabama State 
Board of Education in December 1999. Tliose standards reference specifically Knowledge 
and Skills Teachers Need to Deliver Effective Reading Instruction, a document developed by 
the ARI and published by the Alabama State Department of Education in February of 1998. 

• A frequently mentioned impact of tire ARI on pre-service teacher education programs was the 
effectiveness of pre-service teacliing experiences in Literacy Demonstration Sites where 
students could observe, complete field experiences and internships, and become a part of 
research-based, effective practice. 

Limitations to the Evaluation 

Tire results of the Year Two Evaluation of tire Alabama Reading hhtiative must be 



interpreted within tire context of several conditions: 



Student outcome data came almost exclusively from the Stanford Achievement Test - Ninth 
Edition (Stanford 9), a nomi-referenced measure tliat forms the accountability system in 
Alabama. Tliree items are worth considering when interpreting results: 

• The primary purpose of a norm-referenced Lnstmment is to compare achievement of 
individuals or groups to that of the nomiing group. Nomi-referenced achievement 
instmments such as the Stanford 9 are not created as direct measures of the literacy level 
of the reader. 

• Since the Stanford 9 is tlie accountability instmment for Alabama, the evaluation of the 
ARI is occurring within a competitive enviromnent where all schools in Alabama are 
expected to perfomi to a state-established standard on this test. The fact that ARI schools 
made greater gains than those found in non-ARI schools is noteworthy given the efforts 
that all Alabama schools are making to improve achievement on the Stanford 9. 

• Given tlie nature of nonn-referenced tests, factors such as regression toward the mean 
and insensitivity to small differences or gains could account for some variability in gains 
reported for ARI schools. 

Tlie tests currently used to assess early literacy skills of students in kindergarten through 
second grade are not designed to assess ongoing development of reading ability during the 
first three years in school. Tlierefore, the evaluation does not present information concerning 
the progress of students in Grades K-2, where progress might be expected to occur at a 
greater rate tlian in upper grades. 

All infonnation from survey data was self-reported and was not verified by on-site 
observations. 

A specific model was used to identify liigher-performing and lower-performing schools. It is 
likely that other models using different selection criteria would, to some degree, identify 
different subsets of higher- and lower-perfonning schools. Therefore, conclusions regarding 



differences between liiglier-performing and lower-perfomiing schools should not be 
generalized beyond the model used for tliis evaluation. 

Recommendations 

Based on the findings of the evaluation as well as the limitations discussed above, the 

evaluators make the following recommendations to the ARI and subsequent evaluations of the 

initiative: 

1 . Place greater emphasis on the central role of the principal in facilitating the effective 
implementation of ARI. 

2. Work to increase the helpfulness of all reading specialists. Additionally, seek ways of 
increasing the number of reading specialists, since a prerequisite to “helpfulness” appears to 
be tlieir availability to work directly with strugglmg readers and with teachers 

3 . Develop strategies to increase the direct involvement of higher education partners in 
assisting classroom teachers. Additionally, there is a need for creative incentives tliat reward 
service to ARI schools within the traditional higher education faculty evaluation framework 
of service, scholarsliip, and teaching. 

4. Explore ways of providing continued professional development, support, and 
recognition to schools beyond tlie first year of implementation. 

5. Increase, to the extent possible, the use of criterion-referenced measures that are 
designed to document progress of students toward reading at or above grade level. 
Implementing fall and spring testing using the individual or group form of the criterion- 
referenced reading assessment currently administered in kindergarten through Grade 2 and/or 
linking these measures psychometrically would permit the use of available instalments for 
tliis purpose. 

6. Include longitudinal data on “matched” students across grades within schools in future 
evaluation efforts. 
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7. Include case studies of the higher-achieving and lower-achieving ARI schools that 
include observation data as well as interview data. 

8. Require the collection of other school outcome measures such as discipline referrals, 
special education referrals, and library circulation so that such data can be reported by all 
schools participating in the ARI. 

Summary and Discussion 

Limitations notwithstanding, overall findings include small but positive differences in 
Stanford 9 reading results favoring ARI schools. Tlie decrease in the percentage of struggling 
readers and the increase in the percentage of grade-level readers also favored ARI schools over 
non-ARI schools. These results provide evidence of tlie extent of ARI’s success in addressing 
both tlie expansion of reading power and effective intervention. However, as noted in the 
limitations, the assessment of early literacy skills presented problems; therefore, ARI’s efforts to 
address beginning reading could not be documented using student outcomes. 

Several other positive results of ARI emerged from both outcome data and the reports 
and data provided by constituency groups. Additional student assessment findings from the 
evaluation included concomitant changes in other Stanford 9 subtests, including Total Math, 
Total Reading, Science, and Social Science. Furthermore, among those schools reporting such 
data, systematic and substantial decreases were found in discipline and special education 
referrals. Teachers reported increased use of research-based strategies, increased student reading 
time, greater confidence and enthusiasm toward instruction, greater awareness of .strug glin g 
readers, an improved learning environment, improved student reading and writing skills, and 
improved student motivation to read. Higher education partners and reading faculty reported 
changes in course content as well as improved clinical experiences of pre-service teachers. 

Despite overall improvements exhibited on the Stanford 9, there was considerable 
variability among ARI schools in the progress they demonstrated. An examination of survey 
results and their relationship to school outcomes revealed three factors that discriminated between 
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higher- and lower-performing school ■ clative to reading achieve” nent. These factors included (a) 
the principal as a champion of ARJ; !- ) the hands-on, helpful reeding specialist; and (c) tlie 
deeply involved higher education pa- tncr. Other factors included the influence of professional 
development, student reading time, mention to instmctional stralv gies, ongoing assessment, and a 
focus on stmggling readers. Succe;-. in reading, which equates lo success in school and in life, is 
a result of many factors, some of wh ich may be interdependent. With the leadership of tlie 
principal, support from the reading specialist and from higher ediication partner, and professional 
development activities, teachers through research based, effectixe practice were able to create an 
atmosphere for learning that helped readers at all ability levels bLCome better readers. “America 
will become a nation of readers when \ erified practices of the best teachers in tlie best schools 
can be introduced throughout tlie country” (Anderson, et. al., 1985. p. 120). 
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